Statistical parsing with non-local dependencies

Non-local dependencies occur when a head (e.g. a verb) and its dependent (e.g. an argument) is separated by some intervening material which does not depend on the same head. Because non-local dependencies play an important role in determining predicate–argument structure of sentences, identifying them correctly is essential for Natural Language Processing. However, this task appears to be rather difficult, and for this reason, non-local dependencies have received only limited amount of attention in the statistical parsing literature. This thesis explores the problems identifying non-local dependencies poses for statistical parsing technology. We argue that the difficulties are due to the enlarged search space, an effect of the large amount of locally unresolvable ambiguities introduced by non-local dependencies. We show that the search space can be efficiently reduced by taking lexical and local information into account. We present several simple parsing models incorporating this knowledge. We claim that non-local dependencies in English can be efficiently and accurately recovered by an appropriate combination of shallow approaches. In particular, we show that a finite-state machine without explicit knowledge of phrase structure information is able to detect heads participating in non-local constructions with state-of-the-art accuracy. This machine is employed to constrain the search space of a phrase-structure parser. The parser, when coupled with the finite-state preprocessor, is fast and achieves the best reported results on recovering non-local dependencies. The accuracy of the system crucially depends on the way the parser and the preprocessor are combined. We develop a novel probabilistic framework where a preprocessor guides a parser through imposing soft constraints on the search space. The parser takes not only the hypotheses of the finite-state machine into account, but also incorporates the preprocessor’s probability estimate for these hypotheses, and thus improves its own estimate for the given structure. This combination method attains our goals: the system is simple and, at the same time, efficient and accurate.